Skip to content

Add some v0.3.0 API testing in daily integration tests - #1358

Open
bharat-thotakura wants to merge 82 commits into
mainfrom
bt/update-daily-tests
Open

Add some v0.3.0 API testing in daily integration tests#1358
bharat-thotakura wants to merge 82 commits into
mainfrom
bt/update-daily-tests

Conversation

@bharat-thotakura

@bharat-thotakura bharat-thotakura commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

Closes #1354

bharat-thotakura and others added 7 commits March 5, 2026 16:56
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura
bharat-thotakura marked this pull request as ready for review March 5, 2026 23:29
@bharat-thotakura
bharat-thotakura marked this pull request as draft March 6, 2026 03:06
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura
bharat-thotakura marked this pull request as ready for review March 10, 2026 02:22
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura
bharat-thotakura marked this pull request as draft March 10, 2026 05:31
bharat-thotakura added a commit that referenced this pull request Mar 18, 2026
Presently, the daily integration test workflow uses only 2 workers from
`pytest-xdist` and generally can take a while to finish, waiting on some
slower tests relative to others. Anticipating
#1358, which [will
slow
down](https://github.com/Infleqtion/client-superstaq/actions/runs/22884111478)
the workflow even more, this PR adds `pytest-split` to break up the
(current) total of 44 unit tests from the `css` and `qss` integration
tests combined into 4 concurrent workflows, each using 2 workers. (Note:
the tests are split up evenly across the 4 workflows (called 'groups'),
but `pytest-split` can unevenly group time-consuming tests if given
`pytest` timing info)

Additionally, on a more speculative & hopeful note, this PR also applies
`pytest-split` on the three notebook checks in the regular CI to
continue to mitigate against
#1129 (with the
idea being that each concurrent workflow should now only assign 1
notebook per worker (of course, subject to change if new notebooks are
added)). In [initial
testing](https://github.com/Infleqtion/client-superstaq/actions/runs/22887601930/job/66403774452),
this did not seem to offer much in time reduction due to the small
number of notebooks being tested to begin with (& the worker overhead
perhaps), but this approach might still be worth experimenting with.

Lastly, to confer any potential speed benefits, this PR also updates all
relevant CI jobs to use Python 3.14 from the previous Python 3.13

---------

Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Co-authored-by: richrines1 <85512171+richrines1@users.noreply.github.com>
bharat-thotakura and others added 3 commits March 18, 2026 11:05
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura

bharat-thotakura commented Mar 19, 2026

Copy link
Copy Markdown
Contributor Author

An example integration test workflow against this branch can be found here

@bharat-thotakura
bharat-thotakura marked this pull request as ready for review March 19, 2026 16:31
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura bharat-thotakura changed the title Add some initial v0.3.0 API testing in daily integration tests Add some v0.3.0 API testing in daily integration tests Aug 3, 2026
@bharat-thotakura
bharat-thotakura marked this pull request as ready for review August 3, 2026 23:37
bharat-thotakura and others added 2 commits August 3, 2026 18:41
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (8)

qiskit-superstaq/qiskit_superstaq/daily_integration_test.py:80

  • _get_validated_list_compiled_circuits calls out.compiled_circuits(idx) for every index and then calls out.compiled_circuits() again. This is redundant and can increase job polling/refresh overhead; retrieving the list once and validating it is sufficient.
        assert isinstance(out, qss.SuperstaqJobV3)
        assert all(
            isinstance(out.compiled_circuits(idx), qiskit.QuantumCircuit)
            for idx in range(num_circuits)
        )
        compiled_circuits = out.compiled_circuits()

cirq-superstaq/cirq_superstaq/daily_integration_test.py:82

  • _get_validated_list_compiled_circuits calls out.compiled_circuits(idx) for each index and then calls out.compiled_circuits() again. Fetch the list once and validate its contents/length to avoid redundant job refresh/polling calls.
        assert isinstance(out, css.JobV3)
        assert all(
            isinstance(out.compiled_circuits(idx), cirq.Circuit) for idx in range(num_circuits)
        )
        compiled_circuits = out.compiled_circuits()

qiskit-superstaq/qiskit_superstaq/daily_integration_test.py:60

  • In the JobV3 path, compiled_circuits() / compiled_circuits(0) are called multiple times; even if cached, this adds redundant work and makes debugging failures harder. Cache the result once and assert on it.

This issue also appears on line 75 of the same file.

    assert isinstance(out, qss.SuperstaqJobV3)
    assert isinstance(out.compiled_circuits(), list)
    assert len(out.compiled_circuits()) == 1
    assert isinstance(out.compiled_circuits(0), qiskit.QuantumCircuit)
    return out.compiled_circuits(0)

qiskit-superstaq/qiskit_superstaq/daily_integration_test.py:47

  • provider fixture assumes request.param is always set; if a future test uses the fixture without indirect parametrization this will raise at collection/runtime. Consider providing a default API version (e.g., the existing v0.2.0 behavior) to make the fixture safer to reuse.
    api_version = request.param
    return qss.SuperstaqProvider(api_version=api_version)

qiskit-superstaq/qiskit_superstaq/daily_integration_test.py:122

  • This hard-coded exclusion list will be easy to forget and could mask regressions for these targets. Add a TODO describing the removal condition (and ideally link to a tracking issue) so it’s clear when this should be deleted.
        # Temporary filtering of targets without target info:
        if backend.name not in ("aqt_demo_qpu", "aqt_iqm20q_qpu"):
            assert backend.target_info().get("target") == backend.name
            assert backend.target.num_qubits is not None

cirq-superstaq/cirq_superstaq/daily_integration_test.py:49

  • service fixture unconditionally reads request.param; if another test later uses this fixture without indirect parametrization it will fail. Consider defaulting to v0.2.0 so the fixture remains usable without extra decorators.
    api_version = request.param
    return css.Service(api_version=api_version)

cirq-superstaq/cirq_superstaq/daily_integration_test.py:62

  • In the JobV3 path, compiled_circuits() / compiled_circuits(0) are called repeatedly. Cache the list once and validate against it to avoid redundant work and make failures clearer.

This issue also appears on line 77 of the same file.

    assert isinstance(out, css.JobV3)
    assert isinstance(out.compiled_circuits(), list)
    assert len(out.compiled_circuits()) == 1
    assert isinstance(out.compiled_circuits(0), cirq.Circuit)
    return out.compiled_circuits(0)

cirq-superstaq/cirq_superstaq/daily_integration_test.py:283

  • This hard-coded target exclusion can easily linger and hide regressions for those targets. Add a TODO describing when it can be removed (and ideally a tracking issue link).
        # Temporary filtering of targets without target info:
        if target_name not in ("aqt_demo_qpu", "aqt_iqm20q_qpu"):
            assert service.target_info(target_name).get("target") == target_name

@bharat-thotakura

Copy link
Copy Markdown
Contributor Author

Integration test passing against latest commits

bharat-thotakura and others added 8 commits August 4, 2026 11:09
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@dowusu-antwi
dowusu-antwi self-requested a review August 12, 2026 15:42
@bharat-thotakura

Copy link
Copy Markdown
Contributor Author

Note: update some tests to be less brittle against failures seen in workflows like this

@CLAassistant

CLAassistant commented Aug 26, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration test failed

3 participants